1. Field of the Invention
The present invention relates to a three-dimensional image photographing system for generating three-dimensional image information from luminance information and distance information.
2. Description of the Related Art
Heretofore, as methods for measuring the shape of an object there have been known a passive method (shape from X, stereo view) and an active method (laser radar, pattern projection, spot projection). The passive method is a general-purpose method and is characteristic in that there are few restrictions on an object, while the active method is characteristic in that the measurement accuracy is generally high, but that the measurable range is small due to, for example, a limit encountered in an employable projector part.
In a pattern projection method, which is an active method, pattern light as reference light is projected to an object and a projected pattern is photographed in a direction different from the pattern light projected direction. The photographed pattern is deformed in conformity with the shape of the object. Three-dimensional measurement of the object can be done by correlating the photographed deformed pattern with the projected pattern.
Such correlation between the deformed pattern and the projected pattern should be done in a simple manner while minimizing the occurrence of error. This is a subject to be attained in the pattern projection method. In an effort to achieve this subject, various methods for pattern projection have been proposed (e.g. space pattern coding, moire, color coding). For example, in Japanese Published Unexamined Patent Application No. Hei 5-332737 is disclosed a shape measuring system of a small size and a simple structure, which uses a space coding method. The system disclosed therein comprises a laser beam source, a lens system for shaping the laser beam into a slit shape, a scanner which radiates the shaped laser beam to an object and scans the object, a CCD camera which detects reflected light from the object, and a controller for controlling those components. When the object is scanned with the laser beam with use of the scanner, there occur a portion radiated with the laser beam and a portion not radiated with the same beam. If the radiation of the laser beam is performed in accordance with a plurality of different patterns, the object will be divided into N number of distinguishable portions. By checking in which divided portions the pixels on images are contained, the shape of the object can be determined by calculation, which images are obtained by photographing the object at different positions with use of CCD cameras. In this case, for enhancing the resolution, scanning with laser beam and photographing with cameras are needed each plural times. For example, for dividing the screen into 256 regions, it is required to perform photographing eight times. Consequently, photographing is difficult for an object in quick motion. In addition, while scanning is performed, it is necessary to fix the photographing system firmly. Thus, it is difficult to effect easy photographing although the system itself may become simplified in structure.
For example, as a method for reducing the number of pattern projection, a color coding method is disclosed in Japanese Published Unexamined Patent Application No. Hei 3-192474. In the color coding method, q and k are each assumed to be a predetermined natural number of 2 or more, and using q or more number of colors, there is projected a coded pattern so that two adjacent light slits are not of the same color and so that the arrangement of colors based on k number of adjacent light slits appears only once, then slit colors are detected from an observed image and slit numbers are obtained. Then, an irradiation direction in each slit is calculated from the slit numbers, whereby distance can be calculated in the same way as in the space coding method. The color coding method involves the problem that the amount of calculation for code restoration is large because the code restoration is made from the arrangement of code row. For example, when division is to be made into 256 regions using three colors of red, green and blue, it is necessary to know the arrangement of eight slit lights around slits. Thus, this method is suitable for only the measurement of an object whose shape permits continuous and long slit observation.
As a method which facilitates the restoration of slits and which projects a coded pattern by a single operation, there is known such a space pattern coding method as is proposed in Japanese Patent Publication No. 2565885. According to this method, there are provided three or more kinds of gradation areas using light and shade of a ternary value or more, or three or more colors, or a combination of light and shade with color. Also provided is a multi-valued lattice plate pattern disposed so that at least three kinds of gradation areas contact one another at an intersecting point of boundary lines of the gradation areas, and a main code, which conforms to the kind and order of gradations, is applied to an intersecting point of projected images resulting from projection of the pattern to an object. This main code, or a combined code obtained by combining the main code with a main code at a surrounding intersecting point, is used as a feature code for identification of the previous intersecting point.
In the conventional methods, however, a certain object to be photographed causes the collapse of coding, making it impossible to effect correlation of correct codes. For example, when a row of patterns projected by a light source is photographed with a camera, drop-out or inversion may occur in the photographed pattern row. A certain shape or reflectance of an object may also cause a change in both the projected pattern row and photographed pattern row, making correlation difficult.
In the color coding method, this problem is avoided by omission of decoding with respect to patterns which are likely to cause drop-out or inversion of slits. In the space pattern coding method, a two-dimensional pattern is used to diminish the aforesaid possibility of error, but a certain object gives rise to the same error basically. Thus, according to the conventional methods, under general photographing conditions not making any limitation on an object, the deterioration of accuracy is unavoidable, although high accuracy is obtained in special conditions set within a laboratory or under a limited condition of an object.
According to the method using a light source to effect projection, when an object having a wide range is to be photographed, a three-dimensional shape will not be obtained with respect to a portion incapable of being covered by a projected pattern. Also as to a shade area formed when a projected pattern is shut out by an object, a three-dimensional shape is not obtained because it is impossible to make the measurement of distance.
Accordingly, it is an object of the present invention to provide a three-dimensional image photographing system capable of photographing a three-dimensional image with high accuracy without dependence on an object.
It is another object of the present invention to provide a three-dimensional image photographing system capable of obtaining both luminance information and distance information simultaneously.
The above objects are achieved by a three-dimensional image photographing system comprising a projector for projecting a coded pattern, a first camera for photographing the projected pattern in an optical axis direction of the projector, and a second camera for photographing the projected pattern in a direction different from the optical axis direction of the projector, wherein a new code is allocated to a region where the amount of change of the photographed pattern obtained by the first camera relative to the projected pattern is not smaller than a predetermined value, then using the allocated code, first distance information is generated from the photographed pattern obtained by the second camera, and a three-dimensional image is obtained on the basis of the first distance information and luminance information obtained by the first camera.
As to a region where the amount of change of the photographed pattern obtained by the first camera relative to the projected pattern is smaller than the predetermined value, second distance information can be generated by correlating the luminance information obtained by the first camera and luminance information obtained by the second camera with each other. The second distance information is used for generating the above three-dimensional image. The second camera may be composed of a third and a fourth cameras disposed on both sides of the first camera in the same plane.
The foregoing other object of the present invention is achieved by using as a projector a light source which generates light of the invisible region. In this case, the first camera is provided with a light dividing unit for dividing incident light, a first photographing unit for photographing divided light through an invisible region pass filter, and a second photographing unit for photographing divided light through an invisible region cut-off filter. Without using such a light source, there may be adopted a construction wherein the projection of pattern from the projector is performed every other frame and both photographing of a pattern and acquisition of luminance information are performed alternately frame by frame by the first camera.
A three-dimensional image photographing method according to the present invention comprises projecting a coded pattern, photographing the projected pattern in the same optical axis direction as in the pattern projection and also in a different optical axis direction, allocating a new code to a region where the amount of change of the photographed pattern obtained by photographing in the aforesaid same optical axis direction relative to the projected pattern is not smaller than a predetermined value, then using this allocated code, generating distance information from the photographed pattern obtained by photographing in the different optical axis direction, and forming a three-dimensional image in accordance with the distance information and luminance information obtained by photographing in the same optical axis direction.
According to the present invention constructed as above it is possible to provide a three-dimensional image photographing system capable of photographing a three-dimensional image with high accuracy without dependence on an object and it becomes possible to obtain both luminance information and distance information simultaneously.